[sanitizer_common] [Darwin] Adopt _dyld_get_dyld_header - #182943
Conversation
|
@llvm/pr-subscribers-compiler-rt-sanitizer Author: Andrew Haberlandt (ndrewh) ChangesThere is an issue on recent macOS versions with rdar://167854578 Full diff: https://github.com/llvm/llvm-project/pull/182943.diff 3 Files Affected:
diff --git a/compiler-rt/lib/sanitizer_common/sanitizer_procmaps_mac.cpp b/compiler-rt/lib/sanitizer_common/sanitizer_procmaps_mac.cpp
index 979729f15aa16..0e80f2e3916b2 100644
--- a/compiler-rt/lib/sanitizer_common/sanitizer_procmaps_mac.cpp
+++ b/compiler-rt/lib/sanitizer_common/sanitizer_procmaps_mac.cpp
@@ -176,7 +176,7 @@ void MemoryMappingLayout::Reset() {
// The dyld load address should be unchanged throughout process execution,
// and it is expensive to compute once many libraries have been loaded,
// so cache it here and do not reset.
-static mach_header *dyld_hdr = 0;
+static const mach_header *dyld_hdr = 0;
static const char kDyldPath[] = "/usr/lib/dyld";
static const int kDyldImageIdx = -1;
@@ -244,14 +244,18 @@ extern intptr_t _dyld_get_image_slide(const struct mach_header* mh);
extern int dyld_shared_cache_iterate_text(
const uuid_t cacheUuid,
void (^callback)(const dyld_shared_cache_dylib_text_info *info));
+SANITIZER_WEAK_IMPORT const struct mach_header *_dyld_get_dyld_header(void);
} // extern "C"
-static mach_header *GetDyldImageHeaderViaSharedCache() {
+static const mach_header *GetDyldImageHeaderViaSharedCache() {
uuid_t uuid;
bool hasCache = _dyld_get_shared_cache_uuid(uuid);
if (!hasCache)
return nullptr;
+ if (&_dyld_get_dyld_header != nullptr)
+ return _dyld_get_dyld_header();
+
size_t cacheLength;
__block uptr cacheStart = (uptr)_dyld_get_shared_cache_range(&cacheLength);
CHECK(cacheStart && cacheLength);
diff --git a/compiler-rt/lib/sanitizer_common/weak_symbols.txt b/compiler-rt/lib/sanitizer_common/weak_symbols.txt
index 77e7b5d9f702e..600ab8a7c649c 100644
--- a/compiler-rt/lib/sanitizer_common/weak_symbols.txt
+++ b/compiler-rt/lib/sanitizer_common/weak_symbols.txt
@@ -10,3 +10,4 @@ ___sanitizer_symbolize_demangle
___sanitizer_symbolize_flush
___sanitizer_symbolize_set_demangle
___sanitizer_symbolize_set_inline_frames
+__dyld_get_dyld_header
diff --git a/compiler-rt/lib/tsan/go/buildgo.sh b/compiler-rt/lib/tsan/go/buildgo.sh
index d9e56402ad48f..1340071819fcb 100755
--- a/compiler-rt/lib/tsan/go/buildgo.sh
+++ b/compiler-rt/lib/tsan/go/buildgo.sh
@@ -165,7 +165,7 @@ elif [ "$GOOS" = "netbsd" ]; then
"
elif [ "$GOOS" = "darwin" ]; then
OSCFLAGS="-fPIC -Wno-unused-const-variable -Wno-unknown-warning-option -mmacosx-version-min=10.7"
- OSLDFLAGS="-lpthread -fPIC -fpie -mmacosx-version-min=10.7"
+ OSLDFLAGS="-lpthread -fPIC -fpie -mmacosx-version-min=10.7 -Wl,-U,__dyld_get_dyld_header"
SRCS="
$SRCS
../rtl/tsan_platform_mac.cpp
|
|
✅ With the latest revision this PR passed the C/C++ code formatter. |
ea41f5f to
bdf57bf
Compare
|
Fixed clang-format |
With llvm/llvm-project#182943, the race detector syso has a weak import of __dyld_get_dyld_header, which is only defined on newer macOS (26.4+). For external linking with a pre-Xcode 26.4 C toolchain, we need to tell the C linker to permit that symbol not being defined. Pass a flag to do so. Change-Id: I95a3cd2c7fd3ad50bc47985b3ecca0d4e8352162 Reviewed-on: https://go-review.googlesource.com/c/go/+/755261 LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: David Chase <drchase@google.com> Reviewed-by: Lasse Folger <lassefolger@google.com>
|
/cherry-pick 2e7d07a |
|
I think we should cherry-pick this to make sure that clang 22 supports ASAN on macOS 26.4. |
Error: Command failed due to missing milestone. |
|
/cherry-pick 2e7d07a |
|
/pull-request #188913 |
…t mode DirectBuild: deps simple enough to list files compile as first-class ninja edges instead of a cmake sub-process. tinycc converted — drops the overlay CMakeLists.txt and the recurring ASAN workarounds on the c2str host tool, which now gets -fno-sanitize=all unconditionally. ASAN dyld shim: macOS 26.4's Dyld.framework reimplemented dyld_shared_cache_iterate_text in Swift with a _Block_copy that deadlocks ASAN init (llvm/llvm-project#182943). Shim interposes a non-allocating version using _dyld_get_dyld_header. Linked via LC_LOAD_DYLIB + @rpath so it auto-loads with bun-debug — no env var. Darwin+ASAN only. workarounds.ts: self-obsoleting registry. Every temporary fix registers an expectedToBeFixed predicate; configure fails with cleanup instructions once the upstream fix ships. build.ts: quiet build output when exec args present (original bd behavior), `--` separator for flag disambiguation, `--target` accepts space-separated value, colored target name in done message, signal exit re-raised. Arg routing documented in file header. tty.ts: centralized ANSI/TTY abstraction. nameColor() hashes names to stable colors; stream.ts and the done message share it. config.ts/tools.ts: clangVersion captured from the existing toolchain-resolve --version spawn, exposed on Config. findTool returns {path, version}. Also: - configure() no longer prints; build.ts decides based on quiet mode - provides.sources deps get phonies pointing at compiled .o files - scripts/bd, scripts/bd.ps1 removed (bd is a package.json script) - scripts/build/CLAUDE.md: architecture doc with goals, ninja primer, common tasks, arg routing
(cherry picked from commit 2e7d07a)
past the ASAN runtime fix (llvm/llvm-project#191039). MODULAR_ORIG_COMMIT_REV_ID: 1773da5eabdc1456e1fac443ccf0aac439c6a895
…macOS (#34508) ## Problem `bun bd` aborts at startup on macOS releases older than 26.4 (seen on 15.6.1, Homebrew llvm@21): ``` AddressSanitizer: CHECK failed: sanitizer_procmaps_mac.cpp:214 "((res)) == ((0))" (0xffffffffffffffff, 0x0) <empty stack> Abort trap: 6 ``` The post-link `bun-debug --revision` smoke test dies with this, so the debug build never completes. Workaround has been `bun run build --asan=off`. ## Cause `scripts/build/shims/asan-dyld-shim.c` interposes `dyld_shared_cache_iterate_text` to keep ASAN init from deadlocking on macOS 26.4 (llvm/llvm-project#182943). It looks up the private `_dyld_get_dyld_header` via `dlsym` to synthesize the one cache entry ASAN needs. Apple's own `dyld_priv.h` marks that symbol ["Added in macOS/iOS 26.4"](https://github.com/apple-oss-distributions/dyld/blob/dyld-1376.6/include/mach-o/dyld_priv.h) (first appears in dyld-1376.6; absent from dyld-1340 and every earlier tag). On any macOS < 26.4, `dlsym` returns `NULL`, the shim returns `-1`, and compiler-rt's `CHECK_EQ(res, 0)` aborts. The shim is linked into every `cfg.darwin && cfg.asan` build with no runtime check, so it breaks every debug build on macOS hosts that predate 26.4. ## Fix When the shim can't synthesize the entry (`dlsym` returned `NULL`, or no shared cache), delegate to the real `dyld_shared_cache_iterate_text` instead of returning `-1`. The deadlock and the getter were introduced together in 26.4, so "getter absent" is exactly "real iterate is safe". This is the same shape as the upstream compiler-rt fix (llvm/llvm-project#188913): weak-import `_dyld_get_dyld_header`, use it when present, otherwise fall through to the existing iterate path. Calling the original by name from inside the interposer does not recurse. dyld explicitly adds an identity-mapping tuple for the defining image; from [DyldRuntimeState.cpp](https://github.com/apple-oss-distributions/dyld/blob/dyld-1378/dyld/DyldRuntimeState.cpp): > `// now add specific interpose so that the generic is not applied to the interposing dylib, so it can call through to old impl` (This is also how Apple's own `DYLD_INTERPOSE` wrapper example in [dyld-interposing.h](https://github.com/apple-oss-distributions/dyld/blob/main/include/mach-o/dyld-interposing.h) works.) ## Verification Shim is only compiled for `darwin && asan` (no CI lane exists for that combination); verified the change compiles clean with `-Wall -Wextra -fblocks` against stubbed darwin headers. Behaviour on 26.4+ is unchanged: `dlsym` finds the symbol there and the same synthesize path runs as before. ## Why this is the right fix The alternative is gating shim emission on host macOS version in `shims.ts`, but the runtime check is strictly better: one binary works on both, and it matches upstream exactly. The shim is temporary regardless (self-obsoletes via `workarounds.ts` once LLVM 22.1.4+ is the floor; #34299 deletes it as part of the LLVM 22 bump), so the goal here is just to stop breaking pre-26.4 macOS until that lands. <!-- robobun:evidence:begin --> --- **no test proof** · iteration 0 · build/CI scripts only; test-proof not applicable <!-- robobun:evidence:end -->
…macOS (#34508)
## Problem
`bun bd` aborts at startup on macOS releases older than 26.4 (seen on
15.6.1, Homebrew llvm@21):
```
AddressSanitizer: CHECK failed: sanitizer_procmaps_mac.cpp:214 "((res)) == ((0))" (0xffffffffffffffff, 0x0)
<empty stack>
Abort trap: 6
```
The post-link `bun-debug --revision` smoke test dies with this, so the
debug build never completes. Workaround has been `bun run build
--asan=off`.
## Cause
`scripts/build/shims/asan-dyld-shim.c` interposes
`dyld_shared_cache_iterate_text` to keep ASAN init from deadlocking on
macOS 26.4 (llvm/llvm-project#182943). It looks up the private
`_dyld_get_dyld_header` via `dlsym` to synthesize the one cache entry
ASAN needs.
Apple's own `dyld_priv.h` marks that symbol ["Added in macOS/iOS
26.4"](https://github.com/apple-oss-distributions/dyld/blob/dyld-1376.6/include/mach-o/dyld_priv.h)
(first appears in dyld-1376.6; absent from dyld-1340 and every earlier
tag). On any macOS < 26.4, `dlsym` returns `NULL`, the shim returns
`-1`, and compiler-rt's `CHECK_EQ(res, 0)` aborts.
The shim is linked into every `cfg.darwin && cfg.asan` build with no
runtime check, so it breaks every debug build on macOS hosts that
predate 26.4.
## Fix
When the shim can't synthesize the entry (`dlsym` returned `NULL`, or no
shared cache), delegate to the real `dyld_shared_cache_iterate_text`
instead of returning `-1`. The deadlock and the getter were introduced
together in 26.4, so "getter absent" is exactly "real iterate is safe".
This is the same shape as the upstream compiler-rt fix
(llvm/llvm-project#188913): weak-import `_dyld_get_dyld_header`, use it
when present, otherwise fall through to the existing iterate path.
Calling the original by name from inside the interposer does not
recurse. dyld explicitly adds an identity-mapping tuple for the defining
image; from
[DyldRuntimeState.cpp](https://github.com/apple-oss-distributions/dyld/blob/dyld-1378/dyld/DyldRuntimeState.cpp):
> `// now add specific interpose so that the generic is not applied to
the interposing dylib, so it can call through to old impl`
(This is also how Apple's own `DYLD_INTERPOSE` wrapper example in
[dyld-interposing.h](https://github.com/apple-oss-distributions/dyld/blob/main/include/mach-o/dyld-interposing.h)
works.)
## Verification
Shim is only compiled for `darwin && asan` (no CI lane exists for that
combination); verified the change compiles clean with `-Wall -Wextra
-fblocks` against stubbed darwin headers. Behaviour on 26.4+ is
unchanged: `dlsym` finds the symbol there and the same synthesize path
runs as before.
## Why this is the right fix
The alternative is gating shim emission on host macOS version in
`shims.ts`, but the runtime check is strictly better: one binary works
on both, and it matches upstream exactly. The shim is temporary
regardless (self-obsoletes via `workarounds.ts` once LLVM 22.1.4+ is the
floor; #34299 deletes it as part of the LLVM 22 bump), so the goal here
is just to stop breaking pre-26.4 macOS until that lands.
<!-- robobun:evidence:begin -->
---
**no test proof** · iteration 0 · build/CI scripts only; test-proof not
applicable
<!-- robobun:evidence:end -->
Fixes #826. ## Summary Wire rules_cc's stock `lsan` feature into the same augmentation path as asan, ubsan, and tsan: - add `//toolchain/config:use_lsan` and include it in `use_common_sanitizer` - add the Darwin `libclang_rt.lsan_osx_dynamic.dylib` filegroup and route it through `dynamic_runtime_lib` - add an end-to-end `//:lsan_test` and run it with the Linux sanitizer tests - add a focused macOS CI job using LLVM 22.1.8 The Darwin version pin is deliberate. On current macOS, LLVM 19.1.7, 20.1.8, and 21.1.8 all run `main` and then deadlock during LSan's exit check. The leak-check thread calls `dyld_shared_cache_iterate_text`, which lazily loads dyld introspection code and allocates while LSan holds its allocator lock. LLVM compiler-rt #182943 switched this path to `_dyld_get_dyld_header`; that fix was backported to LLVM 22, and LLVM 22.1.8 exits normally. The configured action graph now contains a `SolibSymlink` whose declared input is `libclang_rt.lsan_osx_dynamic.dylib`, so the runtime is provisioned rather than merely found incidentally in the local execroot. ## Verification On macOS arm64: - LLVM 19.1.7: times out after completing `main` - LLVM 20.1.8: times out after completing `main` - LLVM 21.1.8: times out after completing `main` - LLVM 22.1.8: `//:lsan_test` passes - `//:sanitizer_combo_flags_test` passes - `actionlint` and `git diff --check` pass Upstream fix: llvm/llvm-project#182943
…OS 26.4+ (#68595) ### What problem does this PR solve? Issue Number: N/A Related PR: apache/doris-website#4180 (the macOS build guide change that goes with this PR) Problem Summary: **Context.** On macOS the only entry point for the BE unit tests is `run-be-ut.sh`, and it defaults to `BUILD_TYPE_UT=ASAN`. Since macOS 26.4 that default cannot start a single process: every binary linked against llvm.org's `libclang_rt.asan_osx_dynamic` deadlocks during runtime initialisation, before `main()`. `env.sh` selects that same toolchain on macOS (`llvm@20`), so a developer who follows the documented setup gets a build that hangs rather than one that fails. How the macOS toolchain gets chosen matters for the rest of this description: - **Selection.** `env.sh` writes `custom_env_mac.sh`, which puts `$(brew --prefix)/opt/<formula>/bin` on `PATH` for every formula in its `CELLARS` list. Unless `DORIS_CLANG_HOME` is already set, `env.sh` then takes the first `clang` on `PATH` and derives `CC`/`CXX` from it. - **Installation.** `CELLARS` only selects; it installs nothing. Developers install Homebrew formulae by following the macOS build guide, and each CI workflow installs from its own `brew install` list. - **Third-party libraries.** `thirdparty/installed` comes from one of two places: - the published `doris-thirdparty-prebuilt-darwin-*.tar.xz`, which `apache/doris-thirdparty`'s `build-target.yml` builds by running `thirdparty/build-thirdparty.sh` from apache/doris master; - a local run of that same script. `build.sh` also starts one by itself when `thirdparty/installed` lacks a library it checks for. **1. The problem, and what it cost** *The deadlock.* `sample` on the stalled process shows the whole chain: ``` __malloc_init (libsystem_malloc) <- early libSystem init calls malloc_default_zone() `- wrap_malloc_default_zone (asan runtime) `- AsanInitFromRtl -> AsanInitInternal -> InitializeShadowMemory `- MemoryRangeIsAvailable -> MemoryMappingLayout::Next -> get_dyld_hdr() `- dyld_shared_cache_iterate_text_swift <- macOS 26.4 reimplemented this in Swift `- _Block_copy -> malloc `- __sanitizer_mz_malloc (asan's own malloc) `- AsanInitFromRtl() <- re-enters init `- StaticSpinMutex::LockSlow <- spins on the lock it already holds ``` The cause is an OS-side change, not a Doris one. It also does not mean ASAN is broken on macOS 26: Apple's own clang sanitizer runtime runs fine on the same host, and only llvm.org's compiler-rt is affected. Upstream fixed it by weak-importing `_dyld_get_dyld_header` and using it, when present, instead of walking the shared cache (llvm/llvm-project#182943, main `2e7d07a`; backport #188913, release/22.x `7b6514c`). **The fix shipped only in 22.1.8.** 20.1.8 is the last 20.x release and 21.1.8 the last 21.x, so no version of `llvm@20` can ever be made to work. *Why it looks like a hang and not an error.* - `be/CMakeLists.txt` reaches `storage/index/ann` unconditionally, and that directory's `cmake-protect` target calls `add_subdirectory()` on `contrib/openblas`. - OpenBLAS runs an instrumented `getarch` probe from its **configure** step (`contrib/openblas/cmake/prebuild.cmake:1513`, `execute_process(COMMAND .../getarch 0 ...)`), and `execute_process` has no timeout. - So every ASAN build stops at `-- Running getarch` and never moves again. It sits at about 90% CPU and prints no diagnostic. `run-be-ut.sh` on macOS was therefore unusable at its default setting, and the failure gave the developer nothing to act on. *The second obstacle, once the toolchain is bumped.* - clang 22 added `-Wc2y-extensions` and folds it into `-Wpedantic`. - `__COUNTER__` only reached the C standard in C2y (WG14 N3457). `be/src/runtime/runtime_profile.h:73-85` uses it to give two `SCOPED_TIMER` / `SCOPED_RAW_TIMER` expansions on the same line distinct names. - As a result, a clang 22 build of any TU that includes that header fails under `-Werror`. Three representative unity TUs were enough to hit it, so the failure is not confined to one module. *The third obstacle: clang 22 cannot build the third-party libraries.* The first revision of this PR moved every macOS toolchain reference to `llvm@22`, including the third-party build. CI failed both macOS third-party jobs (7m49s, 6m23s) on the first library that exercises a new clang 22 error: ``` unixODBC 2.3.7 src/SQLBrowseConnectW.c:424:82: error: incompatible pointer types passing 'SQLSMALLINT *' (aka 'short *') to parameter of type 'int *' [-Wincompatible-pointer-types] ``` clang 16, clang 20 and Apple clang 21 report this as a warning; clang 22 makes it an error. unixODBC is only the *first* failure: the build aborts there, so every library after it is untested against clang 22. The macOS third-party libraries therefore have to stay on `llvm@20` while the BE moves to `llvm@22`. Leaving the third-party build alone does not achieve that, because it sources `env.sh`: - With `CELLARS` naming `llvm@22`, a machine without `llvm@22` gets a non-existent directory on `PATH`. - `command -v clang` then falls through to `/usr/bin/clang` (Apple clang), which is worse than either explicit choice. - The job that publishes the prebuilt archive in `apache/doris-thirdparty` is exactly such a machine: it installs only `llvm@20` and sets no `DORIS_CLANG_HOME`. *Two latent defects clang 22 then surfaced.* The bump is not a pure version change. clang 22's stricter diagnostics stop the build on two pre-existing bugs, and both are worth fixing on their own merits. | # | Site | Diagnostic | What is actually wrong | |---|---|---|---| | 1 | `be/src/load/group_commit/wal/wal_dirs_info.cpp:98` | `-Wunused-result` | `LOG(INFO) << "… err: {}", e.what();` — the `,` is the comma operator, not an argument separator, so the statement is `(LOG(INFO) << "…{}") , (e.what())`. The `{}` is never substituted, and `e.what()` is evaluated and then discarded: **the error message has never been logged**, only the literal `{}`. A repo-wide scan for the same shape finds exactly this one site (`cloud/src/common/logging.h` matches only inside macro definitions and is not a bug). | | 2 | `common/cpp/sync_point.cpp:208,210` | `-Wthread-safety-analysis` | The function holds `std::unique_lock lock(mutex_)`, then releases and re-takes the mutex with a raw `mutex_.unlock()` / `mutex_.lock()` pair around the callback, which bypasses the lock's ownership tracking. If the callback throws, the re-lock is skipped while `~unique_lock` still believes it owns the mutex, so the destructor unlocks a mutex that is not held (UB) and `num_callbacks_running_` is never decremented. `lock.unlock()` / `lock.lock()` still runs the callback unlocked, as intended, and keeps the ownership state correct on every path. | **2. What this PR does, and why it helps** - **`env.sh`**: the macOS `CELLARS` list moves from `llvm@20` to `llvm@22`, with a comment recording that 22.1.8 is the minimum and why. This is the single line that decides which clang a macOS developer builds the BE with. - **`thirdparty/build-thirdparty.sh`**: on Darwin, right after sourcing `env.sh` and next to the existing environment sanitization, the script unconditionally points `DORIS_CLANG_HOME`, `CC` and `CXX` at `$(brew --prefix llvm@20)` and puts its `bin/` first on `PATH`. If `llvm@20` is not installed, it stops with a `brew install llvm@20` hint rather than falling back to another compiler. - The override has to come after `env.sh` and has to be unconditional. `env.sh` sources `custom_env.sh`, which may export `DORIS_CLANG_HOME` for the BE, and `build.sh` has already exported the BE's `llvm@22` values before it starts this script. - Every macOS third-party build runs this script: `build.sh`'s automatic rebuild, a manual run, the pull request check in `build-thirdparty.yml`, and the `apache/doris-thirdparty` job that publishes `doris-thirdparty-prebuilt-darwin-*.tar.xz`. This one place therefore keeps all of them, and the published archive, on `llvm@20`. - `.github/workflows/build-thirdparty.yml` is unchanged: its macOS jobs already install `llvm@20`. - **`be/CMakeLists.txt`**: add `-Wno-c2y-extensions` for clang 19 and newer, in its own `add_compile_options` call **after** `-Wpedantic`. - The position matters: a later `-Wpedantic` turns the group back on. That is also why passing the flag through `EXTRA_CXX_FLAGS` does not work: that variable lands near the front of the command line. - The version gate matters because the `c2y-extensions` group only exists from clang 19 on. clang rejects an unknown `-Wno-` option like any other unknown warning option (`-Wunknown-warning-option`, fatal under `-Werror`), and this file accepts clang 16 and newer. - For clang 19 and newer the command line is exactly what it was. - **`.github/workflows/be-ut-mac.yml`**: install `llvm@22` for the BE and keep `llvm@20` next to it. - The job builds the BE on top of the downloaded darwin-arm64 prebuilt. When that archive lacks a library `build.sh` checks for, `build.sh` rebuilds the third-party libraries from source, and that rebuild now needs `llvm@20`. The archive falls behind like this whenever master changes the set of checked libraries before the next archive is published. - Both formulae are keg-only, so installing both does not change which one the BE uses. - **`be/src/load/group_commit/wal/wal_dirs_info.cpp` and `common/cpp/sync_point.cpp`**: the two fixes from the table above. Each is one line, the smallest change that removes the defect rather than suppressing the warning. No `-Wno-unused-result` / `-Wno-thread-safety-analysis` is added, because those diagnostics point at real bugs and should keep firing. - **apache/doris-website#4180**: the macOS build guide installs `llvm@22` and `llvm@20` instead of only `llvm@20`. Without it, a developer who follows the guide after this PR has no `llvm@22`, and the BE silently builds with Apple clang. What this PR deliberately does not do is add `llvm@20` to `env.sh`'s `CELLARS`: - `CELLARS` installs nothing. - `build-thirdparty.sh` does not use `PATH` to find `llvm@20`. - The `CELLARS` loop prepends each entry to `PATH`. An `llvm@20` entry after `llvm@22` would put the BE back on `llvm@20` and bring the deadlock back. **3. The classes, and how they call each other** ``` env.sh CELLARS := llvm@22 (selects a compiler; installs nothing) |- generates custom_env_mac.sh: $(brew --prefix)/opt/<cellar>/bin prepended to PATH |- DORIS_CLANG_HOME := dirname($(command -v clang))/.. -> CC / CXX / ASAN_SYMBOLIZER_PATH '- is sourced by: |- build.sh, run-be-ut.sh -> BE: llvm@22 '- thirdparty/build-thirdparty.sh:55 '- :100-110 on Darwin: DORIS_CLANG_HOME / CC / CXX := $(brew --prefix llvm@20) llvm@20 missing -> exit 1, "brew install llvm@20" -> third-party libraries: llvm@20 run by: build.sh (library missing from thirdparty/installed), manual runs, build-thirdparty.yml (PR check), doris-thirdparty build-target.yml (publisher) brew install lists, i.e. who installs which LLVM: macOS build guide (apache/doris-website#4180) llvm@22 llvm@20 .github/workflows/be-ut-mac.yml llvm@22 llvm@20 .github/workflows/build-thirdparty.yml llvm@20 apache/doris-thirdparty build-target.yml llvm@20 be/CMakeLists.txt if (COMPILER_CLANG) |- add_compile_options(-Wpedantic ...) <- enables the c2y group '- if (CMAKE_CXX_COMPILER_VERSION >= 19) add_compile_options(-Wno-c2y-extensions) <- must stay after the line above | be/src/runtime/runtime_profile.h:73-85 '- MACRO_CONCAT(SCOPED_TIMER, __COUNTER__) <- the only __COUNTER__ use in be/src, be/test | be/src/storage/index/ann/cmake-protect/CMakeLists.txt:48 '- add_subdirectory(contrib/openblas) '- cmake/prebuild.cmake:1513 execute_process(getarch) <- where the hang surfaced ``` ### Release note None ### Check List (For Author) - Test: Manual test on macOS 26.5.1 (arm64, dyld-1378), plus CI. - `clang -fsanitize=address` hello world exits 0 with llvm@22 and still deadlocks with llvm@20 on the same host. - `sh run-be-ut.sh` with no environment overrides now selects `Clang-22.1.8` and clears the `-- Running getarch` point that previously hung forever: zero FAILED targets, and `doris_be_test` links. - `sh run-be-ut.sh --run --filter=FormatRoundTest.*` starts the ASAN binary and passes 8 tests. - `-Wno-c2y-extensions` gate, tested on a `__COUNTER__` TU under `-Wpedantic -Werror`: clang 16.0.6 rejects the flag (`unknown warning option`), which is why it is gated; clang 20.1.8 accepts it; clang 22.1.8 fails without it and passes with it. - `build-thirdparty.sh` compiler override, checked under `bash -x` with both `custom_env.sh` and `build.sh` exporting the `llvm@22` values: the effective `CC`/`CXX` and the first clang on `PATH` are llvm@20. With `llvm@20` missing, the script exits 1 with the brew hint before building anything. - `be-ut-mac.yml`: the "Build BE" step script passes `bash -n` on bash 3.2 and 5.3, and its `cellars` array holds both `llvm@22` and `llvm@20`. The workflow runs only on pushes to master and on schedule, so this PR cannot run it. - `build-support/check-build-hygiene.sh` passes. - CI: `Build Third Party Libraries (macOS)`, `(macOS-arm64)` and `(Linux)` pass at `a955357a30f` (run 36542107899), which already contains the `build-thirdparty.sh` override. The first revision, which moved these builds to llvm@22, failed both macOS jobs on unixODBC. - Regression test / unit test: N/A (toolchain, flag and workflow change). - Behavior changed: Yes, for macOS builds only. - The BE builds with llvm@22 instead of llvm@20. - The third-party libraries, including the published prebuilt, stay on llvm@20. Building them from source now requires llvm@20 installed next to llvm@22; without it, the build stops with a `brew install llvm@20` hint. - Existing macOS developers need to run `brew install llvm@22`. Without it, `env.sh` falls back to Apple clang in `/usr/bin` for newly configured build directories. - Does this need documentation: Yes, apache/doris-website#4180. --------- Co-authored-by: Claude Code <noreply@anthropic.com>
There is an issue on recent macOS versions with
GetDyldImageHeaderViaSharedCache, which is fixed by adopting_dyld_get_dyld_header. We weakly declare this to ensure runtimes compile with older SDK (currently on CI bots).rdar://167854578